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fruitful,  lasting  connections  between  scholars  and  practitioners. 
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Abstract 


Many  DoD  Field  Activities  have  been  designated  as  In-Service  Engineering  Agents 
(ISEAs)  for  individual  warfighting  systems.  As  ISEA,  it  is  crucial  to  periodically 
assess  system  capabilities  and  limitations  individually,  as  a  class,  and  as  a  strike 
force,  as  well  as  the  technical  community’s  ability  to  support  the  systems  in  question. 
Issues  resulting  from  these  assessments  must  be  brought  forward  and  addressed  in 
an  appropriate  manner.  From  an  acquisition  perspective,  limitations  that  require 
materiel  solutions  must  be  fed  through  the  Navy’s  budgeting  process  so  that  focused 
and  coordinated  engineering  efforts  can  be  undertaken.  These  issues  must  be 
provided  across  the  acquisition  and  technical  community,  including  the  industrial 
base,  so  that  they  can  influence  change  in  the  design  of  systems. 

An  example  of  this  ISEA  assessment  is  the  Naval  Surface  Warfare  Center  Port 
Flueneme  Division  (NSWC  PHD)  Safety,  Effectiveness,  and  Affordability  Review 
(SEAR)  process.  NSWC  PHD  conducts  In-Service  Engineering  (ISE)  which 
demands  the  rigorous  and  continual  application  of  well-structured,  systematic 
processes  that,  at  their  core,  are  integrated  as  a  Systems  Engineering  approach. 
According  to  the  Acquisition  and  Capabilities  Guidebook,  a  Systems  Engineering 
Plan  (SEP)  is  a  mandatory  milestone  document  that  is  required  at  milestones  A,  B, 
and  C  and  also  program  initiation  for  ships”  (SECNAV,  2008).  One  of  the  biggest 
challenges  to  the  SEP  is  how  to  maintain  the  discipline  for  not  only  the  Milestones, 
but  also  over  the  systems  Lifecycle,  and  to  maintain  the  Safety,  Effectiveness  and 
Affordability  of  each  system.  This  requires  continuous  vigilance,  starting  with  subject 
matter  experts  looking  at  the  reams  of  data  that  support  the  condition  of  the  systems, 
boiling  it  down  to  systemic  trends  that  have  adverse  effect  on  the  mission,  and  finally 
developing  solutions  that  not  only  fix  the  issues,  but  are  affordable  over  the  lifecycle 
and  can  be  fed  back  into  the  design.  This  continuous  loop  of  disciplined  systems 
engineering  rigor  is  the  cornerstone  of  maximizing  mission  performance  capability. 
This  information  must  be  utilized  by  the  acquisition  and  technical  communities  to 
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improve  the  design  and  capabilities  of  our  systems.  This  effort  includes  training, 
logistics,  and  maintenance.  Maintenance  and  upgrade  of  these  programs  is  critical 
to  keeping  warships  relevant,  reducing  obsolescence,  and  increasing  commonality. 
This  will  also  improve  fleet  readiness  and  fleet  interoperability. 

This  paper  will  focus  on  the  process  of  periodic  Safety,  Effectiveness,  and 
Affordability  reviews  at  the  equipment,  system,  platform,  and  battle  group  levels  to 
identify  drivers  that  reduce  readiness  and/or  interoperability  in  the  current  fleet  and 
push  recommended  fix  plans  into  the  Navy  acquisition  process  for  program  updates. 
It  also  will  allow  the  acquisition  and  operational  communities  to  make  more  effective 
and  efficient  decisions  on  funding  affecting  the  Navy’s  budgetary  cycles.  SEAR  data 
provides  a  critical  quantifiable  perspective  to  decision  makers  to  make  better 
acquisition  decisions.  This  information  must  be  shared  with  industry  as  well,  so  that 
they  can  incorporate  it  into  their  planning,  development,  engineering,  manufacturing, 
and  production.  This  facilitates  development  and  promulgation  of  significant  lessons 
learned  across  the  government  and  industry  team,  and  initiates  the  appropriate 
changes  to  technical  procedures,  requirements,  standards,  and  policies. 

Purpose 

The  purpose  of  this  paper  is  to  outline  why  the  Safety,  Effectiveness,  and 
Affordability  Review  (SEAR)  process  is  foundational  and  necessary  for  the  Naval  Surface 
Warfare  Center,  Port  Hueneme  Division  (NSWC  PHD)  to  effectively  execute  its  role  as  an 
In-Service  Engineering  Agent  (ISEA)  and  to  give  a  brief  overview  of  the  process  and  its  core 
elements.  It  will  also  show  how  this  process  needs  to  be  expanded  and  shared  across  the 
other  engineering  agents,  acquisition  community,  and  industry. 

Background 

The  In-Service  Engineering  Agent  (ISEA)  has  always  been  focused  on  the  functions 
of  maintaining  Combat  System  Safety  and  Effectiveness.  In  today’s  environment  of  budget 
constraints,  rapid  introduction  of  commercial  off-the-shelf  (COTS)  equipment,  and 
associated  obsolescence  issues,  Affordability  is  also  a  critical  focus  area.  Not  only  is  a 
system  level  review  of  these  areas  beneficial,  but  the  operational  concerns  with 
interoperability  necessitates  a  platform  level  review. 

NSWC  PHD  has  been  conducting  Safety,  Effectiveness,  and  Affordability  Reviews 
(SEAR)  of  the  Combat  System  Elements  for  which  they  are  responsible  as  ISEA.  They 
have  also  been  conducting  platform  level  SEARs  with  all  of  the  systems  under  their 
cognizance.  The  SEAR  process  is  aligned  with  the  tasking  from  Naval  Sea  Systems 
Command  (NAVSEA)  and  the  Program  Executive  Offices  (PEOs).  This  includes  providing 
the  technical  insight  and  compliance  with  requirements,  standards,  and  policies  with 
recommendations  on  significant  technical  issues.  NSWC  PHD  is  also  tasked  with  facilitating 
the  development  and  promulgation  of  lessons  learned  and  initiation  of  appropriate  changes. 
The  systems  engineering  approach  used  at  NSWC  PHD  could  be  utilized  across  other 
engineering  agents,  as  well  as  industry,  to  provide  a  more  robust  and  holistic  system  and 
platform  level  review  that  would  be  more  beneficial  to  the  program  offices. 

The  quality  and  overall  value  of  NSWC  PHD  contributions  to  the  customer  are 
primarily  reflected  in  the  degree  to  which  the  safety,  effectiveness,  and  affordability  of  in- 
service  weapon  systems,  combat  systems,  ship  classes,  and  carrier/expeditionary  strike 
groups/forces  are  improved  over  time  by  products  and  services  delivered  to  the  Fleet.  The 
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data  and  analysis  generated  in  a  SEAR  provides  valuable  insight  into  the  system  and 
platform  for  the  program  office.  It  would  also  be  valuable  to  the  other  engineering  agents,  in 
particular,  the  Design  Agent  and  the  Original  Equipment  Manufacturer  (OEM).  The  metrics, 
analysis,  and  recommendations  help  maintain  a  viable  system  throughout  its  lifecycle.  It 
also  provides  critical  feedback  into  the  design  and  acquisition  process,  which  could  help 
reduce  Total  Ownership  Cost  (TOC).  To  achieve  the  optimal  return  on  investment  for 
process  and  product  quality  improvements,  SEARs  need  to  be  documented  in  sufficient 
detail  at  the  appropriate  frequency.  The  SEAR  process  is  the  underlying  closed  loop 
systems  engineering  process  that  is  the  backbone  of  NSWC  PHD’s  methods  for  achieving 
the  highest  quality  blend  of  products  and  services. 

The  SEAR  addresses  the  safety  metrics  of  a  system  with  regards  to  risks,  hazards, 
and  incidents.  The  effectiveness  is  reviewed  from  the  perspective  of  the  Probability  of 
Capability  (Pc),  Operational  Availability  (A0),  and  the  People  Factor  (Pp).  The  affordability 
section  relates  the  safe  and  effective  metrics  to  the  cost  as  a  subset  of  TOC.  Cost  drivers 
and  cost  savings/avoidances  are  discussed.  Interoperability  and  System  Integration  are 
reviewed  with  regards  to  the  safety  and  effectiveness  of  the  interfaces  and  relationships 
inside  and  outside  the  lifelines  of  the  system.  Lastly,  metrics  on  how  well  NSWC  PHD  is 
performing  are  reviewed  in  the  areas  of  Fleet  Responsiveness,  Product  Timeliness,  and 
Integrated  Logistics  Assessment. 

Current  State  Approach  and  Challenges 

The  SEAR  Process — The  Foundation  of  In-Service  Engineering 

NSWC  PHD  performs  a  multitude  of  in-service  engineering  roles  encompassing  all 
levels  of  concern  from  the  equipment  level  through  combat  systems,  ship  class  by  warfare 
area,  and  finally,  carrier/expeditionary  strike  groups/forces.  The  SEAR  process  is  the 
disciplined  systems  engineering  method  applied  at  each  level  to  ensure  that  a  consistent 
approach  is  used  to  collect,  analyze,  and  display  data  and  information.  The  goal  is  to 
provide  the  recurring  assessments,  analysis,  and  identification  of  issues  and 
recommendations  to  the  larger  acquisition  community  and  technical  community.  For 
example,  the  SEAR  process  ensures  that  only  authoritative  data  sources  are  employed  and 
that  only  authoritative  requirements  are  used,  against  which  to  measure  the  various  SEAR 
process  metrics.  The  process  also  “cross  pollinates”  practices  across  organization 
boundaries,  resulting  in  best-of-breed  practices  to  be  adopted  by  the  organization  at  large. 

In  order  for  NSWC  PHD  to  effectively  accomplish  its  in-service  engineering  tasking, 
the  systems  engineering  process  must  also  accommodate  aggregating  data  and  information 
for  use  at  the  next  higher  level.  That  is,  a  closed  loop  systems  engineering  process  must 
“roll  up”  data  from  a  combat  system  to  a  warfare  area,  or  from  individual  platforms  of  a  ship 
class  to  the  carrier/expeditionary  strike  group  level.  The  process  must  also  accommodate 
“drill  down,”  that  is,  the  ability  to  reach  underlying  information  that  supports  conclusions 
made  at  a  higher  level.  Only  by  adhering  to  this  process  can  NSWC  PHD  develop  and 
maintain  the  integrated  “big  picture”  needed  to  support  today’s  Fleet. 

By  conducting  the  robust  systems  engineering  analysis,  critical  design 
considerations  are  addressed  throughout  the  acquisition  lifecycle,  which  will  have  major 
impacts  on  TOC  and  will  help  deliver  a  combat  system  with  affordable  operational 
effectiveness.  Acquisition  decisions  and  design  tradeoffs  are  made  with  capabilities  and 
functions  that  affect  system  performance  against  specifications  and  requirements,  often  at 
the  expense  of  technical  effectiveness  and  system  effectiveness.  Fully  understanding  these 
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design  considerations  that  affect  the  reliability,  maintainability,  supportability,  producibility, 
operations,  maintenance,  and  logistics  is  essential  to  improving  the  availability  and 
efficiencies  of  the  system.  This  will  lead  to  reaching  optimal  technical  and  system 
effectiveness.  When  analyzed  in  the  context  of  how  these  affect  lifecycle  cost  and  TOC,  the 
resultant  will  be  a  more  affordable  and  operationally  effect  system  (see  Figure  1).  The 
SEAR  process  provides  the  necessary  systems  engineering  rigor  from  an  In-Service 
perspective  to  be  able  to  provide  positive  impact  on  the  acquisition  lifecycle  and  design  of 
combat  systems. 
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Figure  1.  Important  Design  Consideration  for  Affordable  Operation  Effectiveness 

The  general  guidance  in  conducting  the  SEARs  is  to  analyze  the  system 
effectiveness  based  on  the  design/performance  requirements  or  specifications,  key 
performance  parameters,  key  system  attributes,  and  current  mission  requirements.  The  Pc, 
Pp,  and  A0  parameters  should  be  determined  individually  by  baseline  and  by  block  type, 
depending  on  the  system.  Because  of  the  complexity  in  the  higher  level  platform  SEARs, 
the  analysis,  perspective,  and  approaches  may  be  different.  Issues  at  the  lower  level  may 
be  lost  or  not  visible  when  aggregated,  due  to  redundancies  or  differences  in  higher  level 
requirements.  The  review  must  contain  the  impacts  of  commercial  items  and  non- 
developmental  items.  The  analysis  must  also  take  into  consideration  the  key  logistic 
elements  that  impact  the  areas  being  reviewed.  The  cornerstone  of  the  SEAR  is  to  review 
these  on  an  annual  basis  with  the  engineering  rigor  in  the  analysis  to  identify  trends,  and 
then  to  be  proactive  in  providing  feedback  to  the  technical  and  acquisition  community  in  a 
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closed  loop  process  (see  Figure  2).  The  output  of  the  SEAR  process  provides  NSWC  PHD 
the  ability  to  make  informed  recommendations  and  decisions  required  by  Fleet  and  program 
office  customers.  It  is  also  an  opportunity  to  feedback  to  industry  to  affect  change  and 
improvements  in  the  products  and  services  they  provide  the  government. 
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Figure  2.  Closed  Loop  Engineering  Process 
Brief  Overview  of  SEAR  Process  Elements 
Safety 


Safety,  as  defined  by  the  NSWC  PHD  SEAR  Guide,  is  the  freedom  from  those 
conditions  that  can  cause  death,  injury,  occupational  illness,  damage  to  or  loss  of  equipment 
or  property,  or  damage  to  the  environment.  System  Safety  Programs  are  usually 
established  in  accordance  with  MIL-STD-882E.  In  the  context  of  the  SEAR  process,  it 
represents  the  level  to  which  risks  of  injury  or  damage  (hazards)  to  personnel  and 
equipment  have  been  mitigated,  as  shown  in  Figure  3.  The  safety  element  is  used  to 
validate  the  continued  safety  of  fielded  systems  and  follows  these  steps: 

■  Review  the  process  and  results  of  incorporating  safety  into  the  design. 

■  Identify  those  processes  that  ensure  safety  during  operation  and 
maintenance  of  the  equipment  or  system  in  the  fleet. 

■  Conduct  a  Safety  Stand-Down  prior  to  each  Safety,  Effectiveness,  and 
Affordability  review. 

■  Develop  a  Risk  Matrix  chart  categorizing  each  safety  issue.  The  sample 
chart  (Figure  4)  shows  the  number  of  issues  by  category  and  the  acceptance 
authority  action  required. 
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Figure  3.  Safety  Process 
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Figure  4.  Sample  Safety  Risk  Matrix 

NSWC  PHD,  as  the  Combat  System  In-Service  Engineering  Agent  (ISEA),  needs  to 
maintain  awareness  of  all  safety  issues,  document  them  thoroughly,  and  address  them  in  a 
timely  and  effective  manner.  These  actions  result  in  safer  weapons  systems  and  reduce 
losses  of  life,  limb,  and  property.  The  SEAR  Safety  process  supports  PHD’s  ability  to  make 
informed  assessments  of  Fleet  issues  and  to  share  information  with  all  stakeholders.  It 
helps  promote  best  practices  across  organizational  boundaries  and  maintain  awareness  of 
all  safety  issues  impacting  a  system  or  a  system  of  systems  (SoS)  by  informing  the  fleet  of 
all  safety  hazards  and  initiating  and  implementing  appropriate  corrective  actions. 

Effectiveness 

Effectiveness  is  defined  as  a  function  of  the  Probability  of  Capability  (Pc),  Operational 
Availability  (A0),  and  Probability  of  Personnel  (PP;  also  known  as  the  people  factor).  Each 
aspect  of  Effectiveness  is  analyzed  in  several  of  the  primary  warfare  mission  areas. 
Addressing  more  of  the  primary  mission  areas  for  a  more  holistic  understanding  of  the 
Effectiveness  is  an  opportunity  to  improve  the  process.  An  example  of  the  mission  areas 
covered  in  a  typical  Platform  SEAR  is  highlighted  in  Figure  5. 
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PROJECTED  OPERATIONAL  ENVIRONMENT 


•  The  Multipurpose  Aircraft  Carrier  Mission  is  to  operate 
offensively  in  a  high  density,  multi-threat  environment 
as  an  integral  member  of  a  Carrier  Strike  Group  (CSG) 
or  Expeditionary  Strike  Force  (ESF) 


Primary  Mission  Areas 


Anti-Air  Warfare  (AAW)* 

Anti-Surface  Ship  Warfare  (ASU)* 

Command,  Control  and 
Communications* 

•  Anti-Submarine  Warfare  (ASW) 

•  Command  &  Control  Warfare  (C2W) 

•  Information  Warfare  (IW) 


•  Intelligence  (INT) 

•  Mine  Warfare  (MIW) 

•  Mobility  (MOB) 

•  Missions  of  State  (MOS) 

•  Naval  Special  Warfare  (NSW) 

•  Strike  Warfare  (STW) 

*  Covered  in  the  SEAR 


Figure  5.  Mission  Areas  Covered  in  a  Typical  Platform  SEAR 

Capability:  Pc 

Probability  of  Capability  is  defined  as  the  inherent  ability  of  weapon  system,  combat 
system,  ship  class,  or  carrier/expeditionary  strike  group/force  to  perform  a  mission.  The 
impacts  of  equipment  and  computer  program  reliability  failures  or  human  errors  are  not 
included.  In  the  context  of  the  SEAR  process,  this  definition  recognizes  that  capability  can 
only  be  determined  within  the  framework  of  specific  threats  and  specific  environments. 

The  process  steps  are  as  follows: 

■  Understand  what  the  expectations  are  for  the  system  under  analysis,  as 
defined  in  the  performance  requirements. 

■  Identify  the  current  mission  requirements  and  the  environment  in  which  the 
system  operates. 

■  Address  computer  program  and  interface  limitations  with  other  systems. 

Results  from  the  SEAR  Pc  assessment  are  used  to  identify  areas  of  improvement,  or 
to  highlight  the  lack  of  testing  resources  to  fully  assess  a  mission  area  capability.  The  SEAR 
Pc  assessment  results  are  also  used  to  develop  and  implement  operational  guidance  that 
makes  maximum  use  of  the  inherent  performance  capability,  while  minimizing  the  negative 
impact  of  system  “weaknesses.”  Negative  impact  is  minimized  by  the  ISEA  developing  firing 
guidance,  implementation  of  equipment  or  system  upgrades,  process  improvements,  and 
improvement  of  technical  documentation. 

Operational  Availability:  A0 

Operational  availability  is  defined  as  the  likelihood  that  when  required,  a  system  is 
operating  at  a  pre-defined  performance  level  and  for  a  sufficient  duration  of  time  to 
accomplish  the  mission.  In  the  context  of  the  SEAR  process,  A0  is  examined  as  both  a 
lagging  and  leading  indicator.  The  following  items  are  performed  to  calculate  a  system  and 
platform  A0: 

■  A0  is  calculated  at  the  System/Equipment  levels. 

■  The  measured  values  should  be  compared  to  values  contained  in 
design/performance  requirements  or  specifications  (i.e.,  Weapons 
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Spec/Ordnance  Spec)  for  newer  systems  and  to  past  history  for  all  systems 
(when  available). 

■  For  the  purposes  of  SEAR,  A0  is  calculated  using  critical  and  major  failures 
only. 

■  At  the  platform  level,  the  first  step  is  to  define  the  reliability  block  diagram  for 
the  detect,  control,  and  engage  systems  within  the  combat  system.  An 
example  is  illustrated  in  Figure  6. 

■  Given  the  mission  of  the  platform  and  the  types  of  threats  expected  to  be 
encountered,  an  availability  analysis  is  conducted  given  hard  and  soft  kill 
alternatives  along  with  conclusions  and  recommendations. 


AD  Ao  Model 


Hard 

Kill 


Figure  6.  Example  of  Air  Defense  (AD)  A0  Model  Reliability  Block  Diagram 

Process 

While  NSWC  Corona  is  the  responsible  agent  for  calculating  A0  values,  it  is  the  ISEA 
that  interprets  this  data  and  determines  the  causes  of  availability  trends.  The  communication 
and  relationship  between  NSWC  Corona  and  PHD  are  critical  for  successful  SEAR 
implementation.  The  ISEA  develops  solutions  and/or  provides  recommendations  to  address 
availability  problem  drivers.  Results  of  the  SEAR  A0  assessment  process  can  lead  to  the 
development  of  system  or  equipment  upgrades  that  increase  reliability,  increase 
maintainability,  or  implement  logistics  changes  that  improve  supportability. 

People  Factor:  PP 

Probability  of  Personnel,  or  people  factor,  encompasses  the  human  systems 
integration  factors  in  mission  performance.  It  examines  the  human  performing  all  the 
necessary  steps  on  time  to  properly  maintain,  set  up,  and  operate  one  or  more  systems  and 
complete  the  mission.  In  the  context  of  the  SEAR  process,  the  people  factor  can  be  a 
quantitative  assessment  (ship  manning  levels)  and/or  a  qualitative  assessment  (correct 
NECs  and  proper  training).  PP  is  comprised  of  five  factors  that,  while  often  not  directly 
measurable,  can  be  tracked  for  trend-finding  purposes. 

The  process  steps  are  as  follows: 
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■  Assess  Ergonomics:  Personnel  safety  and  “usability”  of  operation  and 
maintenance  procedures  are  components  of  the  Ergonomics  element. 

■  Assess  Crew  Experience 

■  Assess  Staffing/Manning  Level 

■  Assess  Personnel  Availability 

■  Assess  Training 

Analysis  of  PP  data  can  result  in  findings  within  Personnel  Training  that  result  in  a 
direct  tie  to  other  areas  such  as  Availability  or  related  trends  (proper  system  maintenance  or 
operation,  personnel  qualifications,  manning  levels,  INSURV  results).  The  types  of  metrics 
collected  and  assessments  generated  are  illustrated  in  Figures  7  and  8. 
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Figure  7.  Example  Personnel  Metrics 
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Figure  8.  Example  of  PP  Analysis  on  Manning  Status 
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The  following  are  examples  of  how  the  SEAR  PP  assessment  results  add  value  to 
the  conduct  of  Combat  System  Ship  Qualification  Trials  (CSSQTs),  Test  and  Evaluation, 
and  Fleet  training  events: 

1 .  The  shipboard  training  process  can  be  modified  based  on  pre  commissioning 
unit  (PCU)  training  results  obtained  from  the  Aegis  Training  Readiness 
Center  (ATRC).  Shipboard  training  would  then  be  tailored  to  the  needs  of  that 
particular  crew.  Results  from  at-sea  test  events  can  help  identify 
weaknesses  not  only  in  team  training,  but  also  in  an  individual  sailor  training 
pipeline.  This  can  be  provided  to  the  training  community  for  continuous 
process  improvements. 

2.  As  the  data  from  multiple  exercises  becomes  available,  the  opportunity  to 
make  objective  judgments  on  the  results  of  these  exercises  will  better  reflect 
the  operational  capabilities  of  people  and  systems.  Personnel  manning  and 
formal  training  levels  are  areas  outside  the  direct  control  of  the  ISEA. 
However,  consistent  shortfalls  in  these  areas  can  be  documented  and 
addressed,  via  the  SEAR  PP  process,  to  the  detailing,  training,  and  Program 
Office  communities  to  influence  these  personnel  issues. 

3.  Consistently  weak  areas  should  be  analyzed  from  a  system  standpoint. 
System  requirements  that  overburden  or  are  confusing  to  an  operator  should 
be  addressed  from  a  hardware  or  software  standpoint,  as  required. 
Recommended  changes  can  then  be  documented  and/or  implemented  to 
maintain  and  improve  a  systems  maintenance  and  operational  readiness. 

4.  Deficiencies  that  cannot  be  attributed  to  system  capabilities  can  also  be 
analyzed.  Those  that  have  consistency  across  platforms  should  be  reviewed 
in  relation  to  the  training  process  and  changes  made.  Opportunities  for  these 
changes  are  available  in  the  review  of  formal  training  courses  by  ISEA 
personnel,  ISEA  inputs  to  tactical  and  IETM  documentation  development 
process,  and  the  modification  of  on-site  training  to  ensure  weak  areas  are 
addressed. 

For  those  areas  that  are  random  vice  symptomatic  (one-time  deals),  additional 
training  can  be  provided  as  the  opportunities  in  CSSQT  or  Combat  System  Certification 
evolutions  present  themselves.  The  results  of  these  additional  training  efforts  can  then  be 
documented,  and  changes  implemented  as  necessary  in  technical  manuals  and  in 
documents  providing  operational  guidance,  such  as  Tactical  Memorandums  (TACMEMOs). 

Affordability 

Affordability  is  the  relationship  of  the  SEAR  process  elements  to  cost.  In  the  context 
of  the  SEAR  process,  the  goal  of  examining  affordability  is  to  apply  TOC  methodology  in 
identifying  cost  drivers  and  tracking  cost  trends  that  NSWC  PHD  can  affect. 

The  process  steps  are  as  follows: 

■  Examine  data  utilized  by  the  Fleet  and  Program  Managers  to  assess 
affordability. 

■  Examine  personnel  requirements. 

■  Report  Cost  Savings/Avoidance  Initiatives. 
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Analysis  of  Affordability  data  allows  for  identification  of  trends  in  systems  costs  to  the 
Fleet  and  variances  in  costs  across  various  ship  classes,  baselines,  and  system  variants. 
Examples  are  shown  in  Figures  9  and  10. 
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Figure  9.  Sample  of  Affordability  Data 
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Figure  10.  Example  of  Cost  Savings  Initiatives  Reported  in  a  SEAR 

The  ISEA  adds  value  by  assisting  Program  Sponsors  and  the  Fleet  in  achieving  their 
cost  reduction  goals,  such  as  the  following  improvement  package  that  was  submitted  for 
“Top  Cost”  AN/SPY-1  parts: 

1 .  High  Voltage  Power  Supply  (HVPS)  Inverter  Module:  Ship  change  document 
(SCD)  4603  adds  fault  detection  to  limit  inverter  failure; 

2.  Phase  Shift  Driver  (PSD)  re-design  has  lower  failure  rate;  and 

3.  Electronic  Switch  2-M  repair  authorized. 
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Interoperability/System  Integration 

Interoperability  is  the  ability  of  equipment,  combat  systems,  ship  classes,  or 
carrier/expeditionary  strike  groups/forces  to  provide  services  to  and  accept  services  from 
other  equipment,  combat  systems,  ship  classes,  or  carrier/expeditionary  strike 
groups/forces,  and  to  use  those  services  to  operate  effectively  together  and  to  achieve 
assigned  missions.  System  integration  is  the  progressive  linking  and  testing  of  system 
components  to  merge  their  functional  and  technical  characteristics  into  a  comprehensive, 
interoperable  system.  In  the  context  of  the  SEAR  process,  interoperability  refers  to 
interfaces/relationships  “outside  the  lifelines,”  while  system  integration  refers  to 
interfaces/relationships  “inside  the  lifelines.” 

The  process  steps  are  as  follows: 


■  Collaborative  Assessment  by  NAVSEA  Interoperability  Certification 
Committee  (ICC),  augmented  by  Subject  Matter  Experts  &  Fleet  users; 

■  Reviewed  results  from  land  based  tests  &  underway  events; 

■  Issues  affecting  interoperability  analyzed,  assessed  and  mapped  to  essential 
functions  and  corresponding  mission  areas; 

■  Qualitative  Assessment  based  on; 

■  Quantity  and  severity  of  issues  considered; 

■  Fleet  usage  and  comparisons  with  other  combat  systems; 

■  Technical  results  expressed  in  terms  of  operational  impact/risk;  and 

■  Operational  perspective  provided  by  active  duty  Navy  Joint  Interface  Control 


Officers. 


Analysis  of  System/Platform  Interoperability  and/or  System  Integration  data  allows 
for  identification  of  system  issues  across  various  ship  classes,  baselines,  and  system 
variants,  as  seen  in  Figure  1 1 .  Interoperability  issues  resulting  from  a  typical  analysis  are 
summarized  with  recommendations  for  mitigation.  Resolution  of  these  types  of  issues  will 
require  collaboration  between  multiple  program  offices,  contractors,  and  other  government 
agencies. 

By  identifying,  solving,  or  supporting  the  resolution  of  interoperability/integration 
problems  and  issues,  NSWC  PFID  adds  value  by  bridging  the  gaps  between  stovepipe 
systems,  and  ensuring  systems  operate  as  intended  to  achieve  its  mission.  As  Combat 
System  In-Service  Engineering  Agent  for  surface  warfare,  NSWC  PHD  helps  the  surface 
fleet  work  together  and  with  other  services  and  allies  as  an  effective  instrument  of  our 
national  policy. 
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Figure  1 1 .  Example  of  Combat  Systems  Interoperability  Assessment 

Critical  Success  Elements/Lessons  Learned 

The  critical  success  elements  of  the  SEAR  as  a  systems  engineering  tool  are 
identifying  the  trends  overtime  and  feeding  the  analysis  back  to  the  program  office  and  to 
the  OEM  to  influence  the  design.  The  top  issues  and  cost  drivers  are  identified  with 
proposed  solutions  and  recommendations  for  improvement.  Capturing  the  impact  of  the 
recommendations  in  follow  on  SEARs  requires  coordination  and  feedback  from  the  program 
office,  as  well  as  more  detailed  technical  discussions  with  the  OEM.  When  all  of  this  is  tied 
together,  the  discussion  on  effectiveness  and  affordability  become  powerful  drivers  for 
improvements  with  in-service  systems,  as  well  as  future  designs.  Four  specific  examples 
resulting  from  the  systems  engineering  involved  in  SEAR  include  the  following: 

■  The  identification  of  transmitters  as  a  reliability  driver  for  a  system 
significantly  reduced  TOC  and  workload. 

■  The  SEAR  analysis  for  another  system  led  to  the  justification  of  specific 
tasking  shortfalls  being  identified  and  funded. 

■  The  identification  of  Surface  Warfare  (SUW)  suitability  issues  led  to  a 
collaborative  SUW  white  paper  provided  to  the  PEO. 

■  The  focus  on  interoperability  led  to  follow  on  white  papers  and  concepts 
being  promoted  across  PEOs  and  NAVSEA  with  some  already  being  funded. 

These  examples  show  that  the  lessons  learned  from  the  SEAR  process  are  folded 
back  into  the  acquisition  and  technical  communities.  The  results  of  SEAR  help  address 
many  of  senior  leadership’s  goals,  such  as  achieving  acquisition  excellence,  reducing  TOC, 
improving  quality,  building  the  future  force,  maintaining  our  warfighting  readiness,  getting  the 
requirements  right,  and  making  every  dollar  count. 
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Identified  Transmitters  as  a  Reliability  Driver  for  a  System 

Mean  Time  Between  Failure  (MTBF)  and  Mean  Logistics  Delay  Time  (MLDT)  were 
consistently  the  most  significant  drivers  for  low  A0.  Deeper  analysis  at  the  subsystem  levels 
showed  that  the  transmitter  was  also  one  of  the  most  costly  items.  Thus,  it  was  frequently 
not  on  board  when  they  failed,  resulting  in  long  logistics  delays,  in  addition  to  the  low  MTBF. 

The  reliability  data,  together  with  the  influence  of  the  Fleet’s  Troubled  Systems 
Program,  were  used  to  justify  the  upgrade  to  a  new  transmitter  with  solid  state  technology, 
as  well  as  to  upgrade  the  entire  system  to  a  more  open  architecture.  The  Solid  State 
Transmitter  ORDALT  was  projected  to  reduce  the  cost  of  ownership  by  60%  with  a  savings 
of  $83, 000/ship/year.  With  concurrent  implementation  of  this  ORDALT,  the  annual  Planned 
Maintenance  System  (PMS)  workload  is  projected  to  be  reduced  by  68%  to  78%  depending 
on  ship  class.  The  corresponding  annual  maintenance  cost  will  be  reduced  by  59%  and 
77%  for  these  ship  classes,  respectively.  The  failure  rate  of  the  solid  state  transmitter  was 
reduced  by  94%,  thus  dramatically  increasing  the  reliability. 

The  SEAR  Analysis  for  Another  System  Led  to  the  Justification  of  Specific  Tasking 
Shortfalls  Being  Identified  and  Funded 

During  the  late  nineties,  allowed  onboard  spares  for  one  system  were  reduced  by 
50%,  which  greatly  increased  Mean  Logistics  Delays.  In  addition,  the  ISEA  budget  was 
reduced  from  approximately  $4  million  per  annum  to  less  than  $1  million  per  annum.  By 
working  with  the  Program  Manager  and  the  Fleet,  NSWC  PHD  drafted  a  stop-work  message 
for  the  PM  to  disseminate.  In  the  meantime,  the  Fleet  was  able  to  convince  OPNAV  to 
restore  the  budget  to  previous  year’s  level,  given  the  alternative  of  no  future  ISEA  support. 

Identified  SUW  Deficiencies  During  Pc  Assessment 

During  the  Pc  assessment  of  a  recent  SEAR,  the  analysis  identified  concerns  with 
ships’  forces  not  adhering  to  technical  guidance  provided  to  ensure  that  circuit  card 
assemblies  (CCAs)  are  configured  to  maintain  Combat  System  Computer  Program 
Certification  status. 

As  indicated  in  Figure  12,  all  of  the  ships  surveyed  do  not  meet  the  configuration 
requirements  that  are  needed  in  order  to  maintain  Combat  Systems  Computer  Program 
certification  status.  Most  of  these  ships  are  in  a  de-certified  status  until  CCAs  are  configured 
in  accordance  with  technical  guidance.  This  finding  is  influencing  the  design  of  future 
combat  systems  by  reducing  the  number  of  configurable  items.  New  technical 
documentation  is  being  developed  to  help  with  troubleshooting  efforts  from  a  system  of 
systems  perspective.  Current  technical  documentation  does  not  provide  detailed  guidance 
to  troubleshoot  complex  system  issues.  This  is  leading  to  improving  the  training  curriculum 
to  re-reinforce  the  importance  of  maintaining  configuration  control,  adherence  to  technical 
guidance  and  system  level  troubleshooting  techniques. 
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Figure  12.  SEAR  Pc  Assessment — Configuration  Management  of  Circuit  Card 

Assemblies  (CCAs) 

As  part  of  the  SEAR  process,  a  combat  system  is  currently  assessed  in  two  primary 
warfare  areas:  Air  Defense  and  Surface  Warfare.  The  SEAR  SUW  Pc  assessment  provided 
an  insight  into  areas  that  required  more  attention  in  order  to  improve  our  effectiveness,  Pc. 
Recent  at-sea  testing  results  provided  details  into  how  our  combat  system  elements  are 
integrated  in  a  system  of  systems  architecture.  The  at-sea  integration  of  the  various 
systems,  computer  programs,  and  shipboard  personnel  provides  the  perfect  venue  for 
identifying  and  better  assessing  the  capabilities  and  limitations  of  a  combat  system  in 
support  of  all  mission  areas.  The  SEAR  assessment  provided  visibility  into  some  of  the 
challenges  that  the  technical  community  need  to  address  in  the  following  areas: 

■  Improve  situational  awareness.  These  include  sensor  improvements  as  well 
as  better  information  distribution  throughout  the  ship. 

■  Improve  system  effectiveness  through  improved  sensor  capability  (e.g., 
detection  range,  accuracy,  data  rate  exchange). 

■  Improve  operator  proficiency  through  training  and  other  means. 

■  Improve  weapons  effectiveness  primarily  by  the  introduction  of  a  new  weapon 
or  SUW  mission  for  an  existing  weapon. 

Identified  Interoperability  Issues  and  Recommendations 

The  focus  on  interoperability  led  to  follow  on  white  papers  and  concepts  being 
promoted  across  PEOs  and  NAVSEA.  Highlights  of  this  effort  include  the  following: 

■  Need  for  testing  early  in  the  acquisition  process  and  with  appropriate 
resources  during  at-sea  test  events. 

■  Interoperability  requirements  need  to  be  addressed  at  the  element  level, 
combat  system  and  continue  to  the  system  of  systems  level. 

■  Need  for  conduct  of  a  robust  interoperability  testing  early  during  system 
integration  and  evaluation  phases.  At-sea  interoperability  test  events  are 
conducted  in  a  single  ship  environment.  Interoperability  issues  are  not  found 
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until  a  new  baseline  goes  into  a  multi-ship  environment,  sometimes  after 
deployment. 

■  Need  to  improve  technical  documentation  for  the  operators,  provide  feedback 
to  the  training  community,  and  improve  communication  among  agencies. 

■  Need  for  creation  of  a  set  of  standards  (Interoperability  metrics)  and  practices 
on  testing,  training,  and  evaluation  to  take  the  ambiguity  out  of  the  results. 
Also,  establishing  a  community  infrastructure  for  interoperability  can  help 
resolve  a  great  deal  of  the  conflicts  that  are  present  today. 

■  Increase  the  emphasis  on  interoperability  during  Combat  System  Computer 
Program  certification.  Historically,  for  emerging  Aegis  baselines, 
interoperability  has  not  played  a  role  in  Combat  System  Computer  Program 
Certification.  There  is  a  need  to  place  more  emphasis  on  interoperability 
testing  and  issues  resolution. 

Future  State  Goals  (Technical  and  Programmatic) 

The  future  state  is  envisioned  to  have  all  elements,  equipments,  and  systems  of  the 
entire  combat  system  analyzed  individually,  as  well  as  being  rolled  up  into  a  higher  platform 
level  review.  The  rigor  of  a  systems  engineering  approach  used  in  the  SEAR  process 
should  be  adopted  across  ISEAs.  A  consistent  formalized  process  will  enable  the  technical 
community  to  provide  rapid  and  direct  feedback  into  the  acquisition  lifecycle.  The  goal 
would  be  to  use  the  analysis  and  inputs  from  the  In-Service  community  in  the  acquisition 
and  design  decisions  of  the  program  office,  other  engineering  agents,  and  the  OEM.  This 
would  allow  the  government-industry  team  to  affect  changes  on  current  systems  and  define 
better  requirements  for  designing  future  systems. 

Issues  identified  through  the  SEAR  process  can  support  program  office  investments 
to  improve  their  system  readiness.  The  value  of  this  process  has  not  been  widely 
advertised  outside  of  NSWC  PHD.  Once  mature,  the  SEAR  process  can  provide  metrics 
that  may  indicate  areas  that  ships  should  investigate  prior  to  major  inspections,  exercises, 
and  deployments.  Program  managers  may  benefit  from  metrics  that  highlight  high  cost 
drivers,  maintenance/manning/training  issues,  diminishing  manufacturing  sources, 
reliability/maintainability/availability  concerns,  and  open  safety  issues. 

Another  goal  is  to  introduce  a  Mission-based  SEAR  that  would  show  the  impacts  of 
systems  performance  in  particular  mission  areas,  given  the  various  readiness  decisions 
affecting  funding  levels  and  capability  upgrades.  A  phased  approach  would  be  utilized, 
starting  with  AD  and  SUW  mission  areas  on  a  single  ship,  followed  by  the  addition  of 
Undersea  Warfare  and  Strike.  All  mission  areas  on  multiple  ships  would  lead  in  to  the 
mission  focus  based  on  Operational  and  Tactical  Situations.  This  would  require  more 
extensive  data  from  Materiel  Readiness  Database,  a  new  Casualty  Reporting  database,  and 
the  Operational  and  Tactical  Situations  block  diagram  to  support  analysis. 

Increasing  Scope  to  Include  Other  Warfare  Centers/SYSCOMs  Who  Support  ISEA 
Work  and  Fleet  Sustainment 

The  assessments  used  to  produce  the  data  for  a  SEAR  are  being  conducted  at 
different  levels  including  equipment  and  system.  The  challenge  is  to  take  the  individual 
equipments,  elements,  and  systems  into  a  holistic  platform  level  SEAR.  This  is  partly  due  to 
the  differences  in  analysis  methodology  used  by  different  ISEAs.  The  difficulty  comes  from 
analyzing  and  understanding  the  complexities  of  stacking  or  additive  effects  when  rolling  up 
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the  data.  The  main  thrust  of  these  reviews  is  to  look  at  the  trends  and  issues  vice  the 
individual  metrics. 

Even  though  NSWC  PHD  is  the  ISEA  for  many  of  the  combat  systems,  other  ISEAs 
need  to  provide  inputs  for  their  systems  in  order  to  assess  effectiveness  of  a  platform  or 
strike  group.  For  example,  C4I  systems  need  to  be  assessed  for  impacts  to  interoperability 
within  a  platform,  as  well  as  with  other  platforms  within  a  strike  group.  Electronic  Warfare 
systems  need  to  be  assessed  as  part  of  the  detect  control,  and  engage  missions  within  a 
platform.  Once  the  SEAR  process  is  stabilized  and  tools  are  available  to  facilitate  data 
collection,  discussions  will  be  initiated  with  other  ISEAs  regarding  the  SEAR  process  and 
their  possible  participation  in  the  assessments.  This  would  allow  a  better  understanding  of 
the  element  and  system  level,  as  well  as  a  more  comprehensive  look  at  the  system  of 
systems  and  platform  level.  This  would  be  extremely  useful  across  multiple  program  offices 
and  industry  partners. 

Use  of  a  Database  Collaboration  Tool 

There  are  many  authoritative  databases  which  address  similar  issues  from  different 
perspectives.  A  collaboration  tool  would  compare  data  from  various  sources  to  validate 
ground  truth  or  to  investigate  specific  issues  further. 

Data  collection  and  sorting  has  been  and  still  is  the  most  time  consuming  and 
laborious  part  of  the  SEAR  process.  Analysis  is  the  critical  element  of  SEAR,  which 
oftentimes  is  cut  short  depending  on  the  amount  of  data  collected  and  sorted.  The  data 
collaboration  tool  would  have  the  data  available  in  the  correct  format,  so  that  the  analysis 
can  be  conducted  when  required — not  after  the  time  it  takes  to  collect  data. 

Information  Assurance  Assessment 

Navy  Cyber  Defense  Operations  Command  Computer  Tasking  Order  06-02  (DTG 
062305Z  MAR  06)  requires  all  afloat  platforms  to  conduct  monthly  Information  Assurance 
Vulnerability  (IAV)  scans.  IA  needs  to  be  part  of  the  SEAR  process  and  will  be  added  to  the 
list  of  SEAR  metrics. 

The  ISEA  needs  to  ensure  our  systems  have  the  capabilities  to  Protect,  Detect,  and 
React  to  internal  and  external  attacks  through  the  application  of  security  services  such  as 
the  following:  Availability,  Integrity,  Authentication,  Confidentiality,  and  Non-Repudiation.  We 
need  the  assurance  that  information  is  not  disclosed  to  unauthorized  persons,  processes,  or 
devices,  support  cross  domain  solutions,  multiple  levels  of  protection  for  voice/data  traffic 
and  provide  cryptographic  solutions  interoperability  between  allied/coalition  partners  and  the 
United  States.  In  addition,  attacks  are  expected  to  occur,  and  the  detection  tools  and 
procedures  that  will  allow  the  capability  to  react  to  and  recover,  reconstruct,  and  examine 
the  sequence  of  events  and/or  changes  in  an  event  need  to  be  in  place. 

Predictive  and  Proactive  ISE  Support 

Currently,  the  ISEA  plans  to  ensure  systems  are  available  when  needed.  As  an 
example,  high  failure  rate  critical  parts  are  spared  onboard  ships  to  minimize  down  times. 
The  spares  loadout  and  approved  parts  list  are  developed  using  various  sparing  models. 
This  does  not  prevent  failures  from  occurring — it  just  minimizes  down  time.  The  next  step 
would  be  for  the  ISEA  to  predict  when  a  potential  failure  of  a  critical  part  on  a  ship  will  occur 
and  recommend  that  replacing  the  part  will  prevent  a  system  casualty  while  underway.  By 
knowing  when  critical  parts  are  replaced,  together  with  the  part’s  MTBF,  we  can  develop  the 
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ship’s  mission  profile  and  determine  if  the  part  has  a  high  probability  of  failing  during  a 
particular  mission.  A  collaboration  tool  would  be  required  to  sort  the  large  amounts  of  data 
required  to  predict  potential  failures  that  may  impact  a  ship’s  mission.  As  with  most  of  the 
information  from  the  SEAR,  this  is  critical  to  share  with  the  program  office  and  industry,  so 
that  contracts  can  be  put  in  place  and  the  appropriate  infrastructure  can  be  established, 
allowing  the  Navy  to  buy  the  right  parts  at  the  right  time  with  minimal  delay  time. 

Redesign  the  SEAR  Process  so  That  It  Incorporates  Elements  of  the  Capabilities 
Based  Assessment  Methodology  in  Order  to  Provide  Relevant  Recommendations  to 
Our  Program  Managers 

Definition:  The  Capabilities  Based  Assessment  (CBA)  is  the  Joint  Capabilities 
Integration  and  Development  System  analysis  process.  It  answers  several  key  questions  for 
the  validation  authority  prior  to  their  approval:  define  the  mission;  identify  capabilities 
required;  determine  the  attributes/standards  of  the  capabilities;  identify  gaps;  assess 
operational  risk  associated  with  the  gaps;  prioritize  the  gaps;  identify  and  assess  potential 
non-materiel  solutions;  and  provide  recommendations  for  addressing  the  gaps  (CJCS, 

2009). 

The  SEAR  focus  needs  to  address  “what  it  takes  to  win”  for  a  warfare  area,  major 
combat  operation,  or  a  specific  mission  thread,  and  not  transformation  for  transformation’s 
sake.  The  SEAR  recommendations  need  to  identify  the  optimal  mix  of  assets  (Sensors, 
Weapons,  C2,  Networks,  Platforms,  Warriors)  and  the  functional,  performance,  and 
integration  requirements  and  candidate  solutions  to  support  warfighter  operational 
requirements.  The  SEAR  recommendations  need  to  support  acquisition  decision-making  by 
providing  traceability  back  to  warfighter  requirements.  It  must  also  assess  the  impact  of 
programmatic  trades,  system  failures,  and  engineering  decisions  on  warfighter  capability, 
including  management  of  materiel  and  non-materiel  solutions. 

By  using  the  CBA  methodology,  the  SEAR  will  provide  the  program  manager  with 
recommendations  with  clear  impacts  to  a  warfare  area  requirement,  major  combat  operation 
requirement,  or  specific  mission  thread  requirement.  The  SEAR  recommendation  should 
identify  interoperability  and  other  technical  risk  areas  in  a  systems  architecture  and  the 
impact  of  potential  issues  on  the  operational  mission. 

Warfare  Center  Future  Way  Ahead  Recommendations 

The  way  ahead  for  the  SEAR  should  be  to  adopt  more  robust  systems  engineering 
processes  across  engineering  agents  to  better  assess  the  systems  they  support  not  only  at 
the  equipment  or  system  level  but  at  a  more  comprehensive  platform  level.  The  SEAR 
process  is  the  backbone  and  foundation  of  the  systems  engineering  process  at  NSWC  PHD. 
The  leadership  at  PEO  IWS  has  embraced  this  systems  engineering  approach  and 
recognizes  the  usefulness  and  value  that  it  adds.  Plans  are  in  process  to  expand  this 
process  across  the  ISEAs  to  allow  for  all  elements  and  systems  to  be  rolled  up  into  the 
platform  level  SEARs,  which  would  provide  more  insight  into  the  whole  integrated  combat 
system.  This  approach  will  allow  the  warfare  centers  to  fully  data  mine  and  integrate  the 
information  that  is  already  collected.  Ultimately,  this  would  enable  a  comprehensive  and 
fully  integrated  view  of  Fleet  performance.  It  would  provide  the  necessary  information  to  the 
PEO  to  understand  the  state  of  the  Fleet,  to  identify  affordability  and  TOC  initiatives,  as  well 
as  to  support  the  Navy’s  budgetary  process  by  raising  critical  Fleet  issues.  Industry  can  also 
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use  this  information  to  better  understand  the  requirements,  improve  their  current  products, 
and  incorporate  them  into  future  system  designs  more  effectively  and  efficiently. 

The  objective  of  the  way  forward  in  applying  this  systems  engineering  process  is  to 
provide  detailed  feedback  from  the  Fleet  and  In-Service  community  to  the  program  office 
and  industrial  base,  which  provides  insight  and  helps  drive  well-informed  decisions.  It 
should  provide  valuable  inputs  to  the  future  performance  requirements  and  design 
assumptions,  such  as  manning  levels,  crew  training,  logistics,  and  maintenance  issues. 

This  process  must  maximize  the  use  of  existing  authoritative  data  from  across  the 
community. 

This  paper  supports  the  recommendations  that  the  SEAR  process,  as  a  fundamental 
systems  engineering  tool,  should  be  utilized  by  ISEAs  for  systems  under  their  cognizance. 
As  the  Combat  Systems  ISEA,  NSWC  PHD  will  then  be  able  to  roll  up  the  individual 
equipment,  elements,  and  systems  into  a  more  inclusive  platform  level  assessment.  The 
products  of  the  SEAR  should  be  provided  to  the  PEOs  to  allow  them  to  make  more  informed 
acquisition  decisions  that  will  impact  the  future  design  and  upgrades.  The  information  must 
also  be  shared  among  the  other  Engineering  Agents  and  the  OEM,  so  that  the  information 
from  the  in-service  perspective  can  affect  change  early  in  the  design  process  and 
acquisition  lifecycle. 
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Safety,  Effectiveness, 
Affordability  Review  (SEAR) 
Process  Overview 


□  A  systems  engineering 
process  used  by  the  In- 
Service  Engineering  Agent 
(ISEA)  to  effectively  execute 
its  mission 

□  All  w  I  E  t  m  k  inf  rm 
recommendations  with 
respect  to  readiness,  life  cycle| 
maintenance  and 
modernization 


Closed  Loop  Engineering  Process 


RESULTS 

•  Engineering  0\ange 

•  Teldanieal,Byi9*in-x 

•  NAVMSG^ 

•  Training  , 


r 

DATA/  INFO 

•  47 90/2 K 

•  CASREP 

•TOR 

□  Promotes  the  sharing  of  best 
practices  and  lessons  learned 

□  Key  to  knowledge 
management 


ISEA  ANALYSIS 

•  SE/#d^(|j|j 

•  Investigation 

•  ECP 

•  Help  Desk 


Improves  Fleet  Readiness  from  a  safety,  capability, 
maintainability  and  availability  aspect. 
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SEAR  Process  for  Affordable 
Operational  Effectiveness 


C 


SAFETY 


EFFECTIVENESS 


r> 


INTEROPERABILITY 
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Effectiveness 
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□  Effectiveness  consists  of: 

□  Capability  -  Perform  specific  mission 

□  Availability  -  O.  erational  availability 

□  Personnel  -  Documentation,  Training,  HSI 


Aq  =  Operational  Availability 
Pc  =  Probability  of  Capability 
Pp  =  Probability  of  Personnel 


Expressed  as  a  function  of  Capability,  Availability  and  Personnel 
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Effectiveness:  Availability 
Sample  RMS  Metrics 

Five  1  _ear  .  eriods  each  endin_  JUL31 


Ship  Class  1  Ship  Class  2  Ship  Class  3 

Reliability:  MTB(EME) 


|  Not  Csrned 


i  Not  fn  Stocit 


I  In  frock 


I  Mot  Camed 


!  NoUoSWOk 


I  ask 


I  Not  Gamed 


I  NotlnSttKk 


In  Stock 


Supportability  -  CASREPs  Per  Shipyear  and  Response  Time  (Days) 

 .  .  -   — —  -  .    -  


_ AvpCASftEPTmw  mm  C4  ■»  C3  :  l  Q2  —  Avg  CASREP  Tim*  M  C3  HC2  —  Ava  CASftEP  Tm»  HCl  ■■  C3  HC2 


MTB(EME)  uptrend  &  Parts  Issued  down  trend  &  CASREP  uptrend  =  Ao  downtrend 
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Affordability  Example 
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1800 

1600 
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1200 

1000 

800 

600 

400 

200 

0 


HISTORICAL  PARTS  COST  IN  $1.000s 


FY06  FY07  FY08  FY09  FY10YTD* 

i  iPM  Cost  i  i Fleet  Cost 

•  Part  Cost  per  Ship 
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SEAR:  Interoperability 


H 

Air  & 
ASCM 
Defense 

suw 

usw 

MIW 

EW 

STRK 

AMB 

PCD 

PCD 

PCD 

PCD 

PCD 

PCD 

PCD 

Surveillance 

Track 

Reporting 

N/A 

Identification 

N/A 

N/A 

N/A 

N/A 

Mutual 

Tracking 

N/A 

N/A 

N/A 

N/A 

N/A 

Positively  ID 
Friendly 
Forces 

N/A 

N/A 

N/A 

N/A 

N/A 

Engagement  & 
Force  Status 

N/A 

N/A 

N/A 

Air  Control 
support 

l  |  |  N/A 

N/A 

N/A 
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Lessons  Learned/Recommendations: 

Poor  Reliability 


□  DATA/ INFO 

Graphics  Processor 
exceeds  predicted  Failure 
Rate 

□  ISEA  ANALYSIS 

Various  component  failures 
and  workmanship  issues 

□  RESULTS 

OEM  implementing 
workmanship  and  process 
control  improvements 
through  out  manufacturing 
process. 


WKG-02-03 

AN/UYQ-70(V)  IVS  4100P  GRAPHICS  PROCESSOR 


Summary:  This  item  has  not  met  it's  predicted  MTBF  at  Integration  Sites  in  previous  reporting  periods. 

Following  info  presented  by  ANiUYQ-70  Reliability  Working  Group  at  UYG-70  Aegis  Production  Program  Review  10/22/02: 
Analysis  results  of  vendor  supplied  failure  reports  indicate  various  component  failures  no  failure  trends  identified. 
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Lessons  Learned/Recommendations: 
High  Fleet  Repair  Cost 


□  DATA/ INFO 

OPTAR  costs  for  DD  SWT 
is  high 

□  ISEA  ANALYSIS 

DD  SWTs  are  within 
predicted  failure  rate.  DD 
SWTs  are  reaching  end  of 
life 

□  RESULTS 

ISEA  and  OEM  identified  a 
more  robust  filament  wire 
thereby  doubling  the 
MTBF. 


SWTDD93.M01 


AN/SPY-1  B  TUBE  SWT  D/D  HIGH  FLEET  REPAIR  COSTS  (Y633A) 


Summary  .  The  Fleet  has  reported  that  the  annual  total  repair  (OPTAR)  costs  for  the  D/D  SWTs  (YG33A)  is  high.  A  review  of  the  Fleet  demand  data 
confirmed  the  Fleets  report;  however,  the  usage  rate  is  well  within  predicted  levels. 
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Future  State 


[  Mission  Architectures  ) 


Requirements  Based 
CONOPS 


7 

Schedule 

-■ 

&  Cost 

_ > 

Architectures  Help  Link  CONOPS  Requirements  To  Investment  Decisions 
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Future  State 
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DoDAF / SEAR 


All  View 

•  Information  pertinentto  the  entire 
architecture 


Technical  View 

•  Minimal  set  of  rules  governing  the 
arrangement,  interaction,  and 
interdependence  of  system  parts  or  elements 


Operational  View 


•  Tasks  or  activities  performed,  and  the 
information  that  must  be  exchanged  to 
accomplish  DoD  missions 


frlii 

System  View 

•  System, service,  and  interconnection 

•It  .« 

functionality  providingfor,  or  supporting, 

operational  activities 

_ j 

Safety 

■The  condition  of  being  safe  from  undergoing  or  causing 
hurt  injury,  or  loss 


Capability  of  Performance  (Pc) 

■Capability  to  perform  a  given  mission 


Operational  Availability  (Ao) 

■Likelihood  that,  when  required,  a  system  is  operating  at  a 
pre-defined  performance  level  and  for  a  sufficient 
duration  of  time  to  accomplish  its  mission 

People  Factor  (Pp} 

■Probability  of  humans  performing  all  of  the  necessary 
steps  on  time  to  properly  set  up  and  operate  one  or  more 
systems  and  complete  the  mission 

Affordability 

■Relationship  of  safe  and  effective  metrics  to  cost 
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Benefits  &  Industry  Impact 


Benefits 

□  S  stems  A  roach 

□  Addresses  Fleet  Support 
Concerns 

□  Readiness  Improvement 
Recommendations 

□  Risk  Mitigation 

□  Cost  /  Decision  T radeoffs 

□  Knowled  e  Mana  ement 

□  Facilitates  Reduced  Life 
Cycle  Cost 


□  Industry  Impact 

□  Feedback  through  PEO 

□  Collaboration  via  IPTs  &  WGs 

□  Life  cycle  lessons  rolled  back 
into  Design 

□  New  System,  capability  & 
baseline  development 

□  ID  improvements  in: 

□  Design 

□  Reliability 

□  Life  Cycle  Cost  reductions 

□  HSI 

□  Manufacturing  QA  process 

□  Tools  &  training 
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Way  Ahead 
Recommendations 


□  Expand  this  process  across  the  ISEAs  to  allow  for  all 
elements  and  systems  to  be  rolled  up  into  the  platform  level 
SEARs 

□  Predict  potential  failures  of  a  critical  parts  and  recommend 
replacement  to  prevent  a  system  casualty  while  underway. 

□  Provide  recommendations  with  clear  impacts  to  a  warfare 
area  requirement,  major  combat  operation  requncmeni,  «-»■ 
a  specific  mission  thread  requirement 

□  Data  Anal^  sis  and  recommendations  should  be  ,  rovided  to 
technical  and  acquisition  community,  including  industry 
partners 
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Conclusion 


□  SEAR  process  is  fundamental  to  the  ISEA’s  system  engineering 
process. 

□  Closed  loop  and  disciplineu  piuutjss  mat  is  appneu  iu  tne 
examination  and  internal  sharing  of  data  and  information 

□  Equipment,  Combat  System,  Ship  Class,  and  Strike  Group 

□  Facilitates  integration  of  by  requiring  the  sharing  of  information 
between  the  levels  and  by  promoting  best  practices  across 
organizational  boundaries. 

□  The  SEAR  process  enables  the  ISEA  to  arrive  at  informed 
decisions,  anticipate  Fleet  and  program  sponsor  issues 

□  SEAR  process  provides  technical  and  acquisition  community 
with  recommendations  that  will  improve  fleet  readiness  and 
future  designs. 

Safe,  Effective  and  Affordable  Combat  Systems 
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